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1 Abstract — In this paper, the problem of grid-to-vehicle en- 
^vj ergy exchange between a smart grid and plug-in electric vehicle 
groups (PEVGs) is studied using a noncooperative Stackelberg 
^Ogame. In this game, on the one hand, the smart grid that acts as a 
^ leader, needs to decide on its price so as to optimize its revenue while 
ensuring the PEVGs' participation. On the other hand, the PEVGs, 
which act as followers, need to decide on their charging strategies so 
as to optimize a tradeoff between the benefit from battery charging 
and the associated cost. Using variational inequalities, it is shown 

H - 'that the proposed game possesses a socially optimal Stackelberg 
equilibrium in which the grid optimizes its price while the PEVGs 
\J choose their equilibrium strategies. A distributed algorithm that 
^ enables the PEVGs and the smart grid to reach this equilibrium is 
q proposed and assessed by extensive simulations. Further, the model 
i— —lis extended to a time-varying case that can incorporate and handle 

slowly varying environments. 
t-H ■ 

• Index Terms — Power system economics, smart grids, electric 
I vehicles, game theory, energy exchange, energy management. 
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DUE to the growing concerns for energy conservation and 
the environment, it is expected that plug-in electric vehicles 
(PEVs) 1 will play a major role in the future smart grid (SG) [2]. 
Therefore, several countries are working on establishing novel 
PEV policies and plans because of their significant environmental 
advantages and cost savings [3]. 

The deployment of PEVs will introduce new challenges in 
the design of SGs. These challenges include developing optimal 
charging strategies for the connected PEVs, ensuring efficient 
communications between PEVs and the grid, and managing 
energy exchange between regular loads of the grid and the 
PEVs. In [2], the optimal charge control of PEVs is analyzed 
in deregulated electricity markets based on a forecast of future 
electricity prices and the optimal economic solution for the 

Wayes Tushar is with the School of Engineering, Australian National Uni- 
versity (ANU), Canberra, ACT 0200, Australia. He is also with National ICT 
Australia (NICTA). e-mail: wayes.tushar@anu.edu.au 

Walid Saad is with the Electrical and Computer Engineering Department, 
University of Miami, Coral Gables, FL, USA. Email: walid@miami.edu 

H. Vincent Poor is with the School of Engineering and Applied Science, 
Princeton University, Princeton, NJ, USA. Email: poor@princeton.edu 

David Smith is with NICTA, Canberra, ACT, Australia. He also holds an 
adjunct position at ANU. Email: david.smith@nicta.com.au. 

This work was supported by NICTA. NICTA is funded by the Australian 
Government as represented by the Department of Broadband, Communications 
and the Digital Economy and the Australian Research Council through the ICT 
Centre of Excellence program. 

'PEVs include both battery-only electric vehicles (BEVs) and plug-in hybrid 
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vehicle owner. In [4], a real-time pricing algorithm is proposed 
for smart grids considering smart meter and energy provider in- 
teraction through control messages exchange. A noncooperative 
game model for pricing and frequency regulation in smart grids 
with electric vehicles is studied in [5]. Using facility location 
games, in [6], optimal locations of battery exchange stations were 
derived within a vehicle-to-grid (V2G) network. An optimization 
framework is proposed in [7] for enabling the SG to determine 
the time and duration of the PEVs charging. In [8], a control 
algorithm is developed based on queuing theory to control 
the charging of PEVs. A stochastic programming technique 
is introduced in [9] to show the impact of charging PHEVs 
on a residential distribution grid. The issue of vehicle-to-grid 
(V2G) integration and the prospective communication interface 
to enable this integration are addressed in [10]. In [11], a mean 
field game is proposed to investigate the competitive interaction 
between electric vehicles in a Cournot market consisting of elec- 
tricity transactions to/ from an electricity distribution network. To 
manage a large number of PHEVs at a municipal parking station, 
an algorithm using particle swarm optimization is proposed in 
[12]. An intelligent method for scheduling the usage of the 
available storage capacity from PHEVs and electric vehicles is 
proposed in [13]. Other aspects of electric vehicles in smart grids 
in terms of energy storage, charging and greenhouse gas emission 
reduction are discussed in [14-16]. Beyond PEVs, many possible 
demand side management solutions are also foreseen for the 
smart grid [17] and each of these solutions has a number of 
benefits and cost tradeoffs. Subsequently, it is anticipated that 
many of these solutions, including the incorporation of electric 
vehicles, will co-exist to provide smart energy management for 
the power grid. 

One of the key challenges of widespread penetration of PEVs 
in the power network is the choice of an optimal charging 
strategy for the PEVs. This is mainly due to the fact that the 
integration of the PEVs into the network has a major impact on 
the power grid and can potentially double the average load [4, 
18]. The simultaneous charging of several PEVs in a particular 
area can overload the network and, thus, lead to an interruption of 
services for other consumers. The problem of PEV charging and 
its impact on the power distribution grid and electricity market 
have been addressed in [1-3,7,8] and [19]. However, little has 
been done to develop distributed models and algorithms that can 
capture the interactions between PEVs and the grid, in a grid- 
to-vehicle scenario. There is a need to develop solutions that 
capture the often conflicting objectives between the SG, which 
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seeks to maximize its revenue, and the PEVs, which seek to 
optimize their charging behavior. Because of the limited grid 
capacity and PEVs' energy demands, it is of interest to develop 
a model that can capture the decision making process of the 
PEVs and the grid when the grid's limited energy needs to be 
allocated among the PEVs based on their needs. 

The main contribution of this paper is to provide a com- 
prehensive analytical framework that is suitable for capturing 
the interactions between an SG and a number of PEV groups 
(PEVGs), e.g., parking lots, which must decide on their charging 
profiles. We model the problem as a generalized Stackelberg 
game in which the SG is the leader and the PEVGs are the 
followers. The objective of the PEVGs is to strategically choose 
the amount that they need to charge, so as to optimize a utility 
that captures the tradeoff between the charging benefits and 
the associated costs, given various practical constraints on the 
PEVGs and the main grid. Based on the strategy choices of the 
PEVGs, the leader aims to optimize its price so as to maximize 
its revenues. We analyze the properties of the resulting game 
within the studied model, including existence of an equilibrium 
and optimality, and show that there exists an efficient (i.e., 
socially optimal) generalized Stackelberg equilibrium. We show 
that, due to the coupled capacity constraints between the PEVGs, 
the noncooperative followers' game leads to a generalized Nash 
equilibrium, and the solution enables the capture of not only the 
charging behavior of the vehicles but also the decisions made by ^V - 



grid at peak hours of energy demand e.g., from 12 pm to 4 pm 2 
[20]. The total charging period for the PEVGs is considered to 
be divided into multiple time slots. Each time slot has a duration 
of anywhere between 5 minutes and half an hour based on the 
changing traffic conditions of the PEVs in the group [21]. For 
a particular time slot, the power grid has a maximum energy 
C that it can sell to the connected N PEVGs, allowing them to 
meet their demand 3 . The power grid will set an appropriate price 
p (per unit of energy) for selling its surplus so as to optimize its 
revenue. 

Each PEVG n G M, where N is the set of all N PEVGs, will 
request a certain amount of energy x n from the grid so as to 
meet its energy requirements (e.g., to go back home after office 
work). This demand of energy may vary for the PEVs in the 
group based on different parameters such as the battery capacity 
b n of the PEVG, the available energy in the PEVG's battery at 
the time of plug-in to the grid, the price p per unit of electricity 
and the nature of usage (e.g., two identical PEVGs with different 
travel plans may need different amounts of energy). During peak 
hours, the available energy for servicing PEVGs is often limited 
[20], and, thus, the PEVGs will request only the amount needed 
due to their immediate need for charging. Since the net energy 
C available for the PEVGs at the grid is fixed, the demands of 
the PEVGs must satisfy 

5>„<C. d) 



the SG. We propose a novel algorithm that the PEVGs and the 
grid can use, in a distributed manner, so as to reach the desired 
equilibrium. We also show that the proposed algorithm enables 
the system to adapt to time-varying environmental conditions 
such as arrival/departure of PEVs. Using extensive simulations, 
we assess the properties of the proposed scheme. 

The rest of this paper is organized as follows. Section II 
describes the system model. In Section III, we formulate the 
noncooperative generalized Stackelberg game and we discuss its 
properties. In Section IV, we propose a distributed algorithm 
for finding the equilibrium. Adaptation of the proposed game 
to time-varying conditions is discussed in Section V. Numerical 
results are analyzed in Section VI and conclusions are drawn in 



II. SYSTEM MODEL 

Consider a power system consisting of a single power grid 
(i.e., an SG), several primary and secondary load subscribers or 
consumers and a smart energy manager (SEM). Here, the SG 
refers to the main electric grid which is connected to the area 
of interest via one or more substations. Further, we consider 
that the SG is servicing a certain area or groups of primary 
consumers such as industries, houses and offices. After meeting 
the demands of the primary consumers the grid wishes to sell its 
excess of energy (if any) to the secondary users, such as PEVs, 
connected to it in that area. Consider a number of groups of 
PEVs (hereinafter, we use the term PEVG to denote a group of 
PEVs, acting as a single PEV entity), which are connected to the 



Given the amount requested by the PEVGs, the SG sets a price 
p per unit of energy so as to optimize its revenue from selling 
energy by strategically choosing its price per unit of energy. 
Although the SG can choose the price p within any range to 
maximize its total revenue, a very large p may compel a PEVG 
to withdraw its demand from the grid and to search for alternate 
markets or wait until the prices drop. Therefore, an optimal price 
needs to be chosen by the grid operator which is not very high 
(to avoid losing customers) and not very low (to avoid losing 
revenue), in an effort to maximize its profit. 

To successfully complete the energy trading, the PEVGs and 
the grid interact with each other and agree on the energy 
exchange parameters, such as the selling price and the amount of 
energy demanded, that meet the objectives of the both sides. The 
amount of requested energy x n is determined by the physical 
characteristics of PEVG n as well as by the tradeoff between 
the potential benefit that PEVG n is expecting from buying x n 
and the selling price p of the grid. Moreover, the selling price 
p per unit of energy is strongly dependent upon the amount 
demanded by the PEVGs as well as the number of PEVGs that 
are connected to the grid. This is due to the fact that the amount 
of energy available from the grid is fixed and, thus, as the number 
of PEVGs increases, the amount that each PEVG can acquire 
becomes smaller. As a result, the grid can set a higher price to 
increase its revenues. 

2 The proposed scheme is not only restricted to the considered period but can 
also apply for any time duration. 

3 While this paper is focused on the interactions between PEVGs and the grid, 
when the grid has energy that can be provided to the PEVGs, it can also be 
extended to cases in which there are multiple energy sources beyond the main 
grid. 
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It is clear that the demands of the connected PEVGs are 
coupled through the energy constraint in (1), and are also 
dependent on the physical characteristics and capabilities of 
the PEVGs. Also it is important to note that the price set by 
the grid is dependent on the demand of each PEVG. Thus, 
the main challenges faced when developing an approach that 
can successfully capture the decision making process of both 
the PEVGs and the grid are: i)- modeling the decision making 
processes and the interactions between the connected PEVGs 
in the network given the constraint in (1); ii)- developing an 
algorithm that enables the PEVGs in the network to strategically 
decide on the amount of energy that they will request from the 
grid so as to optimize their satisfaction levels given the constraint 
in (1); and iii)- enabling the grid to optimize its price while 
capturing the tradeoff between the PEVGs' participation and 
revenue maximization. To address these challenges we propose 
a framework based on noncooperative game theory. 

III. Noncooperative generalized Stackelberg game 

A. Game formulation 

To formally study the interactions between the SG and the 
PEVGs, we use a Stackelberg game [22] which is a type of 
noncooperative game that deals with the multi-level decision 
making process of a number of independent decision makers or 
players (followers) in response to the decision taken by a leading 
player (the leader) [22]. Hence, we formulate a noncooperative 
Stackelberg game in which the SG is the leader and the PEVGs 
are the followers. This game is defined by its strategic form, 
T = {{MU{SG}),{X n } neM ,{U n } neM ,L(p),p}, hiving the 
following components: 

(i) The PEVGs in Af act as the followers in the game and 
respond to the price set by the SG. 

(ii) The strategy of each PEVG n G M which corresponds to 
the amount of energy demanded x n G X n from the grid 
satisfying the constraint J2 n x n < C. 

(iii) The utility function U n of each PEVG n that captures the 
benefit of consuming the demanded energy x n . 

(iv) The utility function L(p) for the SG (leader of the game), 
which captures the total profit that the grid can receive by 
selling the surplus energy with price p. 

(v) The price p per unit of energy charged by the SG. 

1) Utility Function of a PEVG: For each PEVG neJV, we 
define a utility function U n (x n , x_ n , s n , b n ,p), which represents 
the level of satisfaction that a PEVG obtains as a function of the 
energy it consumes. Here b n is the battery capacity of PEVG 
n and x n is the requested energy from the grid; s n is the 
satisfaction parameter of PEVG n, which is a measure of the 
satisfaction the PEVG can achieve from consuming one unit of 
energy. This s n may depend on PEVG's battery state at the 
time of plug-in to the grid, the available energy at the grid 
and/or the travel plan of the corresponding PEVG. For example, 
a PEVG 1 having less need for energy than another PEVG 2 
(e.g., due to having a fuller battery) will need less energy than 
PEVG 2 to attain the same satisfaction level (i.e., S2 < s{). The 
energy demand may vary based on the battery capacity and/or 



the satisfaction parameter of each PEVG. The price per unit 
of energy can also affect the demand of the PEVG. Thus, the 
properties that the utility of a PEVG must satisfy are as follows: 
(i) The utility functions of the PEVGs are considered to 
be non-decreasing because each PEVG is interested in 
consuming more energy if possible unless it reaches its 
maximum consumption level. Mathematically, 
{x ri , x_ n , s n , b n j p) 



Sx n 



> 0. 



(2) 



(ii) The marginal benefit of a PEVG is considered as a non- 
increasing function, as the level of satisfaction of the 
PEVGs gradually gets saturated as more energy is con- 
sumed, i.e., 



$ U n (x r , 
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b„ . p) 



5x r , 



< 0. 



(3) 



(iii) Hereinafter, we consider that, for a fixed consumption level 
x n , a larger b n implies a larger U n (x n , x_ n , s n , b n ,p) and 
a larger s n leads to a smaller U n (x n , x_ n , s n , b n ,p)- So 
we have 

3U n (x n , x_ n , s n , b n , pj 



Sb„ 



>0, 



^U n ( K x n , x_ n , s n , b n , p) 



< 0. 



(4) 



(5) 



(iv) The price per unit of energy set by the grid affects the 
utilities of the PEVGs and the utility of a PEVG decreases 
with a higher price. That is, 

SUn{x n ,X—ii,S n ,b n ,p) 



5p 



< 0. 



In this work we consider the particular utility 

Un (x n , x_ n , s n , b n: p) — b n x n 



S n Xn 



P^n t 



(6) 



(7) 



,k?n X k\ and x -n = 

,xn], although many of our results 



where x n G [0, C 

[Xl , X2 j • ■ • ) X n — 1 : X n +1 ) • ■ 

can be generalized. 

From (7), the utility of PEVG n is affected by its battery 
capacity b n - This is due to the fact that a PEVG with higher 
b n will have higher marginal utility and thus, needs to consume 
more energy to reach its maximum satisfaction level [23]. The 
utility also depends on the satisfaction parameter s n of the 
PEVG. PEVGs with the same capacity but with a different 
satisfaction parameter will have different marginal utilities and, 
thus, will be satisfied by different amounts of energy. To this end, 
we assume that a PEVG does not consume any energy beyond its 
maximum satisfaction level, i.e., U n = if x n > (x n * — x™ 1 ), 
where x™ is the initial energy in PEVG's battery at the time 
of plug-in to the grid and x n * is the energy that maximizes its 
utility within the given constraint in (1). 

2) Utility Function of the Power grid: A PEVG n that 
consumes x n MWh of electricity during a designated period of 
time at a rate p per MWh is charged px n which is the cost 
imposed by the SG on the PEVG. The objective of the SG is to 
maximize its revenue by selling the available energy surplus to 
the PEVGs after meeting the demand of its primary consumers, 
and also to control the nature of energy consumption of the 
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PEVGs. While this energy surplus C is fixed, the SG wants to 
set a price p per unit of energy so as to optimize its revenue, 
given the demands of the PEVGs. Thus, we assume the utility 
function for the grid is 

L(p,x n (p)) = py]x n , (8) 

n 

which captures the total revenue of the grid when selling the 
energy required by all PEVGs at a price p per unit of energy. 

In this proposed game, the SG can control the price p per 
unit of energy it wishes to sell. Connected PEVGs respond to 
the price by demanding a certain amount of energy, given the 
constraint in (1), so as to maximize their utilities. Thus, for a 
fixed price p, the objective of any PEVG n is 



and 



max 

E„e(c-x_ r 



s . t. j^- 



< C. 



(9) 



Here, we can see that the amount of energy demanded by each 
PEVG n depends, not only on its own strategies and price but 
also on the demand of other PEVGs in the network through (1) 
and the constraint is the same for all players. This leads con- 
nected PEVGs to engage in a noncooperative resource sharing 
game, which is a jointly convex generalized Nash equilibrium 
problem (GNEP) due to the same shared constraint (1). Note 
that, in game theory, a noncooperative game in which the players' 
actions are coupled solely through the constraints, such as in the 
proposed model, is a special class of games whose solution is the 
generalized Nash equilibrium [24,25], and hence, the proposed 
followers' game, for any price p, is a noncooperative resource 
sharing game whose solution is the generalized Nash equilibrium 
(GNE). Then, given all the PEVGs' demands are at the GNE, the 
leader, i.e., the grid, chooses the price to maximize its revenue. 
Thus, for the given GNE demands of the PEVGs, the objective 
of the grid is 



max L(p) — max px n ■ 



(10) 



Thus, one suitable solution for the formulated game V is the 
Stackelberg equilibrium at which the leader reaches its optimal 
price, given the followers' optimal response at their GNE. At this 
equilibrium, no player (leader or follower) can improve its utility 
by unilaterally changing its strategy. In classical Stackelberg 
games, the followers typically choose their Nash equilibrium 
strategies. In our model, due to the coupled strategies as per 
(1), the PEVGs need to seek a GNE instead of a classical 
Nash equilibrium. To this end, hereinafter, we refer to our game 
as a generalized Stackelberg game (GSG) whose solution is 
the generalized Stackelberg equilibrium (GSE) in which the 
followers reach a GNE. 

Definition 1: Consider the GSG V = 
{(AfUSG),{X n } n ^,{U n } n ^,L(p),p} defined in III-A 
where U n and L(p) are given by (7) and (8) respectively. A set 
of strategies (x*,p*) constitutes the GSE of this game, if and 
only if it satisfies the following set of inequalities: 

Vz„*gx*, n£N^x n <C (11) 



L(p*,x*) >L(p,x*). 



(12) 



Thus, when all the PEVGs' demands are at the GSE, no PEVG 
can improve its utility by deviating from its GSE demand and 
similarly, no price other than the optimal price 4 p* set by the 
grid at the GSE, can improve the utility for the grid. 

B. Existence and efficiency of GSE 

In noncooperative games, the existence of an equilibrium 
solution (in pure strategies) is not always guaranteed [22]. 
Therefore, for our follower game, we need to investigate the 
existence of the GNE in response to a price p. Specifically, we 
are interested in investigating the existence and properties of 
a variational equilibrium (VE), which is a type of GNE to be 
defined below (see also [24]), for our case. This is due to the 
fact that a VE is more socially stable than another GNE (if 
there exists any) and thus, is a desirable target for any algorithm 
to achieve [25]. Particularly for the proposed case, where a 
number of PEVGs in the SG network are demanding energy 
from a constrained reserve, an efficient VE would be the most 
appropriate solution to be considered. Hereinafter, we use VE 
and GNE interchangeably. 

Theorem 1: For a fixed price p, a socially optimal VE exists 
in the proposed game T between the PEVGs connected to the 
grid. 

Proof: First, clearly, by adding the quantity J2 m ^n(PmX m — 
\s m x m 2 ) - J2 m ^nP x ™ to u n in (7) and treating the resulting 
utility function as the new objective function for PEVG n will 
not affect the solution [26]. Thus, the original game is equivalent 
to one in which all PEVGs have the same utility function, 



N 

E 

rn—l 



U(xi, ...,x N ;s 1 ,...,s N ;h,...,b N ;p) 

N 



1 



(13) 



Hence, to determine the socially stable outcome of the game, the 
existence of a solution that maximizes (13) is our main concern. 

Using the method of Lagrange multipliers [27], the Karush- 
Kuhn-Tucker (KKT) conditions for the n th player GNEP is given 
by 



-Vx n U n (x n ,X- n ,s n ,b n ,p) + V x 




where A„ is the Lagrange multiplier for PEVG n. 

First, we note that, for a fixed price p, the followers' game 
admits a jointly convex GNEP, hence, the solution of the GNEP 
with (1) can be found via a variational inequality VI(X, F). This 
essentially reduces to determining a vector z* £ X C K ra , such 

4 The optimal price p* maximizes the utility of the SG for the given GNE 
demand vector x* of the PEVGs in the smart grid network. 
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that (F(z*),z — z*) > 0, for all z G X where X is the set in 
the definition of joint convexity and F(x) = — (V x ll n (x.))^ =1 
[25]. The solution of VI(X,F) is a variational equilibrium. 
Now the KKT conditions can be written as [24] 



F(x) 



AV X ( 



C) = 0, 



C) = 0, A > 0. 



(15) 



Note that the subscript on Lagrange multiplier A is dropped in 
(15) . This is due to the fact that the solution of a jointly convex 
GNEP is a VE if and only if the shared constraint has the same 
multiplier A for all players [24]. 

Now from the definition of F [25], we have 
s\x\ +p — bi 
S2X2 + p - 62 



For solving the monotone VI in our proposed game, we 
consider the Solodov and Svaiter (S-S) hyperplane projection 
method [28, 29]. In the S-S method, two projections per iteration 
are required using a geometric interpretation (see [28]). This 
hyperplane projection algorithm works as follows [28]: Suppose 
we have x k , which is a current approximation to the solution 
of VI(X, F). First the point Proj x [.x fc — F(x k )} is computed 6 . 
Next, the line segment between x k and Pro) x [x k — F(x k )} is 
searched for a point z k such that the hyperplane dH^ := {x £ 



l \{F{z k 



e ) =0} strictly separates x k from any solution 



Sn%n ~t~ V bn 



x* of the problem. A computationally inexpensive Armijo-type 
procedure [30] is used in this S-S algorithm to find such z k . Once 
the hyperplane is constructed, the next iterate x k+1 is computed 
by projecting x k onto the intersection of the feasible set X with 



the hyperspace Hk := {x £ 



(F(z k 



z k ) < 0}, i.e. 



(16) 



Therefore, the Jacobian of F is 
~si 

s 2 



JF 



s r 



(17) 



JF is a diagonal matrix with all positive diagonal elements. 
Hence, JF is positive definite on X, and so, F is strictly 
monotone. Thus, the GNEP admits a unique global VE solution where 
[24]. 

Because of the jointly convex nature of the GNEP the VE is 
the unique global maximizer of (13) [24], which completes the 
proof. ■ 

As a result, from Theorem 1, the GSE, in which the SG sets 
its optimal price in response to the VE demands of the PEVGs, 
admits the socially optimal solution of the proposed game. 

IV. Proposed solution and algorithm 



X n Hk, which contains the solution set [28] [29]. 

Next, we show how the PEVGs reach the VE, for a price p, 
following the optimization of price by the grid when all PEVGs 
are in VE. Then, we detail the algorithm at the end of this section. 

A. GNE for a fixed p 

From (13) and (15), for any PEVG n, the solution of the KKT 
system of variational inequalities is 

b„ - s„x„ - p — A = 0, 



(18) 



A 



- C = 0; A > 0. 



(19) 



For A > 0, the inequality constraint in (19) becomes as equality 
and hence at the VE, 



= C. 



(20) 



In this section, we formulate the GNEP among the followers as 
a variational inequality (VI) problem 5 and propose an algorithm 
that leads to the socially optimal VE. Note that the VE further 
leads to the GSE state of the game as defined in Definition 1. 
Now, we first state the following corollary and then explain the 
solution method for the considered GNEP. 

Corollary 1: The VI associated with the proposed GNEP of 
the connected PEVGs for a price p is a strongly monotone VI 
and thus, the unique VE can be calculated by solving a monotone 
VI. 

Proof: By Theorem 1, we know that the VI associated with 
the proposed GNEP of the connected PEVGs for any fixed price 
p is a strongly monotone VI and the VE is unique. It is shown 
in [24] that the solution of a VE can be calculated by solving 
a monotone VI. Hence, the unique VE solution of the PEVGs' 
GNEP of energy demand within constraint (1) can be calculated 
by solving the strongly monotone VI(X, F). ■ 



Thus, for a fixed p at the grid, the sum of demands of all 
the PEVGs connected to the grid at the VE is equal to the total 
energy C available at the grid. 

At the peak hour of demand, energy in the grid is a scarce 
commodity and, hence, all the PEVGs compete with one another 
for a fair allocation of the available grid energy. Thus, for 
the formulation of the proposed game between the PEVGs the 
available energy at the grid should be less than the total energy 
consumption capacity of the connected PEVGs. This is essential 
for avoiding the trivial case in which all the PEVGs should get 
an allocation equal to their capacity. From (18), we have 
b n - s n x n - p > 

i.e., b n > s n x n +p. (21) 

Taking all N PEVGs connected to the grid into consideration, 
(21) becomes 

]T b„ > P N + (22) 

n n 

which leads to the following proposition: 

Proposition 1: To achieve the maximum utilities at the VE, 
within the constraint in (1), the total capacities of the N grid 



Given X C 



and F 



vector z* G X such that (F(z*), z — z 



\ the VI(X, F) consists of finding a 
) > 0, for all z £ X. 



6 Proj_Y (z) = argmin{||to ■ 



6 X} V2 6 . 
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connected PEVGs must be greater than their total VE demand 
plus a constant equal to pN . 

For the special case in which the PEVGs have different ca- 
pacities (i.e., b n is different for each n) but the same satisfaction 
parameter (i.e., s„ = s for all n € Af), (22) becomes 



Algorithm 1 Algorithm to reach GSE 



6 ™ >PN + sC 

n 

where 

J2 x n = C, from (20). 



(23) 



Now, while Proposition 1 holds for a price p at the grid, from 
(18), the demand of the PEVGs at the VE is given by 

b n - (p + A) 



X n *(p) 



where 



p for any n e Af. 



(24) 
(25) 



B. Price optimization 

Having analyzed the followers' game, we now show how the 
SG can set its optimal price p* given the VE of the PEVGs. 

For the KKT system of Vis described in (18) and (19), the 
selling price for per unit energy is 

p < b n — s n x n . (26) 

Again, from (24), the demand for energy by PEVG n at the VE 
is x n *. So the price per unit of energy satisfies 

p<b n -s n x n *. (27) 

Now, with the condition in (27) and the utility of the SG from 
(8), which is L{p) = p^^x n over p > 0, this dictates that the 

n 

revenue-maximizing price of the grid should be the upper limit 
of (27). Thus, the optimal price 7 of the proposed GSG is 

b n - s n x n *. V (28) 



p 



C. Proposed algorithm 

In order to reach the equilibrium, the PEVGs and the smart 
grid must make their strategy choices with little communication 
between one another. To this end, we propose an algorithm 
that all the PEVGs and the grid can implement in a distributed 
fashion and reach the efficient GSE of the game. We note that, 
in a jointly convex GNEP where F(x) = — (V x E/ n (x))^L 1 is 
strongly monotone, such as our proposed game, the solution of 
the VI converges to a unique VE [24] when the demand of each 
PEVG n is such that the parameter A„ in (15) for all n € Af 
possesses the same value A > 0. In other words, if the parameter 
X n converges to a single value A > for all n £ Af, then x*, 
the demand vector of the PEVGs contains the VE demand of the 
PEVGs. 

For our game, we can use this property and propose an 
algorithm in which each PEVG updates its demand iteratively, 

7 From (25), at p = p* , the slack variable A = 0. Hence \„ converges to 
A = 0, Vn as the game reaches its GSE. 



to the SEM. 



/. Solving VI 

Each PEVG n £ submits its initial demand 
repeat 

a) The SEM checks \ n k for the demand x n k of all 
n 6 Af using (15). 

b) Each PEVG n G M updates its demand x n k using the 
S-S hyperplane projection method [29]. 

S — S method 

i) The PEVG n computes the projection r(x n ). 

ii) The PEVG n updates x n k + 1 equal to r(x rl k ) 

if r(x n k ) = and submit to the SEM. 
otherwise 

iii) The PEVG n determine the hyperplane z n k and 
the half space H n k from the projection. 

iv) The PEVG n updates its demand x n k + 1 from 
the projection of its previous demand x n on to 
X n H n k and submit to the SEM. 

until all X n converge to A > 0. 

The SEM determines the VE demand of the PEVGs. 

2. Optimizing Price 

a) The SEM submits the VE demand of the PEVGs 
to the grid. 

b) The grid optimizes its price p to p* using (28). 
The VE demand and price of the GSG are achieved. 



until all A„ converge to a single value A > 0. In this algo- 
rithm we use the hyperplane projection method to solve the 
proposed VI problem. By using this technique, we guarantee 
that our algorithm always converges to a non-empty solution if 
F is strongly monotone [31] which is always verified in our 
game, as previously shown. Thus, the proposed algorithm is 
guaranteed to converge to a unique solution of the game, given 
the demand constraints of the PEVGs and the grid's capacity C. 
As we explained in Section EH, this convergence implies that the 
proposed GSG reaches its GSE as soon as the grid optimizes the 
price using (28) for the given VE demand of the PEVGs. 

The proposed algorithm uses the S-S hyperplane projection 
method [29] to calculate the demand at the VE for a price p. 
Each PEVG and the SG can implement the proposed algorithm 
to reach the GSE in a distributed fashion with the assumption that 
the SEM can communicate with both the grid and the PEVGs. 
The SEM can use any vehicle to grid infrastructure technique, 
[6], for this communication. As soon as any PEVG n is con- 
nected to the grid, the SEM receives the utility parameters b n 
and s n using V2G. The algorithm starts with the announcement 
of the available energy C and the price per unit energy p by the 
grid. At any given iteration k, in response to this price p, each 
PEVG n updates its demand for a particular amount of energy 
x n k from the fixed amount C of the grid using the S-S method. 
The SEM gets the price p from the grid and checks the parameter 
A„ fc using (15). To enable the SEM to check the value of X n k , 
each PEVG n submits its demand x n to the SEM at the end 
of iteration k. The process continues until all the PEVGs make 
their demands such that A„ = A > Vn. The demand of the 
PEVGs reaches a VE as the SEM determines that A n = A > 
for all n <G Af. Then, the SEM submits the VE demand of the 
PEVGs to the grid and the grid sets the optimal price p* using 
(28) and, thus, the proposed GSG reaches the GSE. 

After the execution of the algorithm the demand of each PEVG 
n reaches its equilibrium value x* n which is given by 



X n (P ) 



(29) 



with the optimal price p* . This is the equilibrium of the game. 
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V. Adaptation to Time-Varying Conditions 

Here, we extend our approach so as to accommodate time- 
varying conditions using a discrete time feedback Stackelberg 
game model with dependent followers [32]. We assume that 
the number of vehicles at a given location, e.g., a parking lot, 
changes gradually in real time with a moderate time duration (for 
example, 5 minutes to 30 minutes) [21]. We also assume that 
the available energy from the grid varies across moderate time 
intervals, e.g., once in an hour [33]. Hence, using a discrete time 
feedback Stackelberg game, we can capture changes in system 
variables from one time instant to another. Prior to discussing 
how the proposed scheme can be adapted in a time- varying 
environment, we define the following parameters: 
Ct : the state variable of the game, which indicates the state of 
available charge at time instant t. 

b l n : the battery capacity of PEVG n, which depends on the 
aggregate quantity of the number of PEVs in the group at instant 
t. 

s*j : the satisfaction parameter 8 of PEVG n at t. 

: the energy demand by PEVG n at t. 
p t : the price per unit of energy at instant t. 
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Fig. 1: Convergence of the demand of each PEVG to the GSE. 
for the SG, and 

* 1 



max Ut 



> m 



max 



J, t 2 



Pt x , 



(32) 



x* <C t at£ = 0,l,2,...,T, 



in the network at time t. 



the vector of demands of all the PEVGs 



(x\,. 



.., x l N ) 



the vector of strategies of 



all PEVGs except PEVG n at instant t. 



L 



T-t 



the payoff function of the grid which 



wants to maximize over the entire peak hour duration. 

u T-t = Et E„ (K x l - \ s n xt n ~ Ptxl) ■ the j omt utility 
function of the PEVGs in the network. 

Consequently, the state transition equation for the time-varying 
system can be defined as [34] 



C t+1 = f t {C t ,pt,x t ), t 



0,1,2,...,T 
T is tl 



1, 



(30) 



where t is an integer time index, and T is the entire peak hour 
duration. For a feedback Stackelberg game, C f is the information 
about the available energy supply at time t, which is gained 
by the grid from (30) and fed back to the PEVGs through the 
SEM [34]. The state of available energy supply at any instant t 
is a function of the demand for energy by the PEVGs, and the 
energy available at the previous time slot. The state transition 
from t to t + 1 takes place when a change occurs either in 
the number of PEVs in the PEVG, or in the available energy 
from the grid. The objective of both the SG and the PEVGs 
is to choose their strategies so as to maximize their utilities in 
each time interval, and thus to maximize their total payoffs in 
the entire time horizon of peak hours. Hence, the problems of 
payoff maximization of the players for a discrete time feedback 
Stackelberg game can be formally expressed as 



for the PEVG n. The objective functions in (31) and (32) refer 
to a feedback Stackelberg game with Nash game constraints 
in the lower level decision making process [32] (similar to the 
j t game proposed in Section III for a single time instant) over the 
whole time horizon. Here, we assume that the leader of the game 
can perfectly gain the information about Ct from (30) [35]. At 
any instant t, the SG gets the information about the volume of 
the parking lot (i.e., and s*" 1 for all n = 1,2, ..,-ZV), the 
demand of the PEVGs in the previous time slot t — 1 as well as 
the per unit price pt-i- Then, it estimates the amount of energy 
that needs to be provided to the PEVGs at t through (30). The SG 
feeds C t back to the PEVGs through the SEM and the PEVGs 
play the jointly convex generalized Nash game of Algorithm 1 
for the allocation of energy within E„ x n — Ct- 



max Lx-t = / ] maxp t : 



(31) 



8 Here, we consider that the satisfaction parameter changes randomly between 
consecutive time slots, for example, due to the random change of vehicles in a 
parking lot from one time slot to the next. 



Here, (x 1 *, x 2 *,..., x T *) and {p{,P2, constitute the 

solution of the discrete time Stackelberg game under a feedback 
information structure with the corresponding state information 
(Ci, C2, Ct), if x** comprises the solution of the GNE 
among the followers for price p^ at each t = 1,2, ...,T [32]. 
The solution will be team optimal if the solution at the GNE 
is optimal, that is if the GNE is a VE [24], for the sub game 
in each t = 1,2, ...,T [32]. Now, given that the Stackelberg 
game described in Section III constitutes a sub-game in each 
time interval t of the feedback Stackelberg game, the sub-game 
will reach its optimal solution (as shown in Section III-B) within 
the constraint J2 n x n ^ @t at eacn * = 1,2, ...,T. Therefore, 
the solution of the discrete time feedback Stackelberg game 
possesses a team optimal solution. 

VI. Numerical analysis 

For our simulations, we consider a number of PEVGs that 
are connected to the grid during peak hours. Here, a single 
PEVG entity represents 1000 vehicles at a specific location [2], 



s 



•o- ©- e -o -o- -o- ©- e -& -o- -o 




- e - 


PEVG 


- 1 


-Q- 


PEVG 


- 2 




PEVG 


- 3 




PEVG 


-4 


- + - 


PEVG 


- 5 



9 . A «6««t«««00 

9" ■ $ " V 



2 3 5 7 10 12 15 

Number of iterations 

Fig. 2: Convergence of the utility of each PEVG to the GSE. 

where each single vehicle is assumed to require 22 kWh for 
every 100 miles [36,37]. The maximum battery capacity of any 
single vehicle is chosen between 150 and 300 miles. Hence, the 
maximum capacity b n of any PEVG n ranges between 35 MWh 
and 65 MWh, which is chosen randomly for each PEVG. The 
available energy at the grid is chosen as 99 MWh. Initially, the 
grid sets the selling price to 17 USD per MWh. We note that the 
chosen parameters correspond to a typical PEVG use case [2, 36, 
37]. However, we duly highlight that these parameters can vary 
considerably according to PEVG usage and type, and economic 
conditions within a given city, or country. 

The satisfaction parameter s n is chosen randomly in the range 
of [1, 2]. The range of s n is chosen based on the assumption that 
the PEVG with the lowest satisfaction parameter s n = 1 will 
get full satisfaction from each unit of energy it will consume 
whereas the PEVG with the highest parameter i.e., s n = 2, will 
reach the same satisfaction level from consuming half the amount 
due to its smaller limitations in terms of its initial battery state 
and travel plan 9 . We do not consider the PEVGs with s n > 2 
as they do not have an immediate need for energy at the peak 
hour. All statistical results are averaged over all possible random 
values of the PEVGs' capacities using around 1000 independent 
simulation runs. 

In Fig. 1 and Fig. 2, we show the demand and the utility at the 
GSE for a network with N = 5 PEVGs. Here, we can see that 
a similar demand by different PEVGs does not always lead to 
a similar utility for the PEVGs. For example, although PEVGs 
2 and 5 in Fig. 1 have almost the same demand at the GSE, 
their utilities are different from one another as shown in Fig. 2. 
This is due to their different battery capacities and satisfaction 
parameters. From the utility in (7), we can see that the maximum 
utility level of a PEVG varies significantly for different values 
of b n and s n . Therefore, with the same energy consumption, 
different PEVGs may obtain a different utility (e.g., PEVGs 2 
and 5). Fig. 1 and 2 show that, after the 10 th iteration, all the 
PEVGs reach their maximum utilities, and, thus, their demands 

'Although a PEVG may not be able to demand the amount of energy equal 
to its total battery capacity due to the scarcity of energy at peak hour, it can ask 
for an amount that must be satisfied in order for its constituent PEVGs to reach 
their satisfactions. 
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Fig. 3: Convergence of the values of A n , Vn to A > as the solution 
of the GSG converges to GSE. 

converge to the GSEt Y 

In Fig. 3, we analyze the convergence speed of the proposed 
algorithm by plotting the values of X n as a function of the 
number of iterations for N = 5 PEVGs. First, recall that 
the demands of all PEVGs converge to GSE for the optimal 
price p* when all the PEVGs reach their VE and at this VE 
A n = A > 0, Vn. Fig. 3 shows that our algorithm converges 
to the GSE after 9 iterations (i.e., the PEVGs reach their VE). 
Hence, as shown by Fig. 3, the convergence speed of our 
algorithm is reasonable. 

In Fig. 4, we show how the price set by the grid converges to 
its optimal value as the strategies of the PEVGs converge to the 
VE for networks of various sizes (N = 5, 10, and 15 PEVGs) 
with an initial SG price set to 17 USD per unit of energy. Fig. 4 
shows that the price converges to an approximate optimal value 
within 5 iterations. This is due to the fact that the SG sets its 
price in response to the demand strategies of the PEVGs. In fact, 
the PEVGs reach an approximate GSE within five iterations, as 
shown for 5 PEVGs in Fig. 1, and, then, they optimize their 
demands within constraint (1). The PEVGs eventually reach the 
unique GSE that maximizes their utilities under this constraint. 
Hence, the price p converges quickly to its optimal value p*. 
Furthermore, Fig. 4 shows that the variation of the grid's price is 
more noticeable when fewer PEVGs exist. This is due to the fact 
that, for a fixed grid capacity, as the number of PEVGs increases, 
there are fewer possibilities of variations in the demands due to 
(1). 

In Fig. 5, we show the effect of the number of PEVGs on 
the optimal price choice of the grid. To do so, we increase the 
number of PEVGs in the network for different grid capacities 
C = 60, 80 and 90 MWh. Fig. 5 shows that the average optimal 
price increases with the number of PEVGs in the network 
because of the increasing energy demand on the SG's limited 
resources. In contrast, increasing the grid's capacity leads to a 
decrease in the optimal price p*. This is due to the fact that, 
as the total available capacity of the grid increases, the grid has 
more energy to sell and, thus, it can decrease its price while 
maintaining desirable revenues. Moreover, for a fixed number 
of PEVGs, as the available energy at the grid increases, the SG 
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Fig. 5: Effect of increasing the number of PEVGs N in the network 
and the grid energy C on the average Stackelberg price. 

reduces its optimal price 10 to encourage the PEVGs to demand 
more energy. 

Fig. 6 shows the average and maximum number of iterations 
needed to reach the GSE of the proposed game with respect to 
the number of PEVGs. In Fig. 6, we can see that, whenever the 
number of PEVGs in the network increases, for the same amount 
of capacity from the grid, the PEVGs require more iterations to 
reach their optimal demands. For example, when the number 
of PEVGs in the network increases from 15 to 25, the average 
number of iterations needed to reach the GSE increases from 52 
to 79. Similar behavior is also seen for the maximum number of 
iterations. 

In Fig. 7, we compare the results of the proposed scheme 
with a particle swarm optimization (PSO) [12] and an equal 
distribution (ED) scheme [39]. In a PSO scheme, a group of 
random solutions 11 (i.e., particles) are scattered over the search 
space, and the particles converge to a near optimal solution after 
a number of iterations. For the PSO algorithm, the particle size is 

10 We assume that the initial price is chosen based on statistical estimation 
such as in [38], so that the grid can maintain a revenue with optimized price as 
the grid capacity increases. 

"in this case the energy demand of a PEVG. 



considered to be 40. The parameters are updated in such a way 
that the constraint in (1) is satisfied. For the ED scheme, the 
available grid energy is distributed equally among the connected 
PEVGs in the network. That is, if the available energy at the grid 
is C MWh and there are TV PEVGs connected to the grid, then 
each PEVG receives an allocation of MWh of energy from 
the grid as long as this allocation does not exceed the maximum 
that a PEVG can be charged. 

Fig. 7 shows that the average utility achieved by the proposed 
scheme is better for most of the PEVGs in the network, except 
PEVGs 9 and 10, when compared to the PSO scheme. This 
is due to the fact that the PSO scheme optimizes the energy 
for each PEVG according to the better particle position in the 
available energy space. This may lead to a better utility for 
PEVGs 9 and 10 at the expense of much lower utilities for the 
rest of the PEVGs. However, the proposed scheme allocates the 
energy among the PEVGs so that the socially optimal solution is 
achieved. Therefore, most of the PEVGs in the network achieve 
improved utilities compared with the PSO scheme. From Fig. 7, 
the proposed scheme has a total utility, on the average, 1.3 times 
the utility achieved by the PSO scheme. Moreover, the proposed 
scheme has, on the average, twice the utility achieved by the ED 
scheme which is a significant improvement. 

Fig. 8 shows the average demand per PEVG as the number 
of PEVGs varies. In Fig. 8, we can see that the average demand 
per PEVG decreases as the number of PEVGs increases. This is 
a direct result of (1) and of the fact that the optimal price set 
by the grid increases as the number of PEVGs increases. From 
Fig. 8, we can see that average PEVG demand for our scheme, 
the PSO scheme, and the ED case decreases as the number of 
PEVGs increases. Fig. 8 shows that the energy demand for the 
proposed scheme is lower than that for both the PSO and ED 
scheme for all N. This can be interpreted by the fact that, in our 
proposed scheme, each PEVG demands a required amount of 
energy within constraint (1) based on its satisfaction parameter 
and available grid energy. Therefore, each PEVG requests a 
socially optimal amount of energy from the grid rather than 
a predefined amount (as in the ED scheme), or near optimal 
amount according to a random search space (as in the PSO 
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Fig. 7: Comparison of the average utilities of the PEVGs at the GSE 
for the proposed scheme, the PSO scheme and ED scheme. 
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Fig. 8: Effect of the number of PEVGs in the network on the average 
demand of a PEVG. 



scheme). Clearly, Fig. 8 shows that the proposed scheme leads 
to better energy utilization than either the PSO or ED scheme. 

In Fig. 9, we show the average utility achieved by all three 
schemes as a function of the number of PEVGs. In this figure, 
we can see that the average utility per PEVG decreases as N 
increases for all three schemes. This is due to the fact that the 
benefit extracted by each PEVG decreases as more PEVGs share 
the fixed available energy from the grid. However, importantly, 
Fig. 9 shows that the proposed scheme has a performance 
advantage at all network sizes which is, on the average, 1.6 times 
that of the PSO scheme. When compared to the ED scheme, the 
proposed scheme shows significant improvements for all network 
sizes, reaching an improvement of up to 3.5 times over the ED 
scheme for N = 25 PEVGs. 

In Fig. 10, we assess the average demand of energy per PEVG 
in a time-varying environment. Here, the state transition of the 
variable Ct in (30) is modeled as an independent stochastic 
process [34], due to the random changes in traffic conditions 
and grid energy from one time instant to another [40] . The state 
of available energy at the grid at any time t is assumed to be 
a uniformly distributed random variable in the range [0.5,1.5] 



Fig. 9: Effect of the number of PEVGs in the network on the average 
utility per PEVG. 

times the average available energy [40]. The capacity of the 

PEVGs bl Vn £ Af at t = 1,2, ,T, is assumed to be a 

uniformly distributed random variable in the range [0.5, 1.5] 
times the average battery capacity [40]. Assuming that the traffic 
conditions in any PEVG changes every 30 minutes, we have run 
independent simulations for allocating energy to the PEVGs for 
eight time slots in peak hours (from 12 pm to 4 pm). The average 
available energy in the grid is 66 MWh (assuming a range from 
33 to 99 MWh), and the variation of energy across time slots 
is modeled by a uniformly distributed random variable between 
0.5 and 1.5 of this amount [40]. Similarly, the random variation 
in PEVGs' capacity at different time slots is captured assuming 
a maximum of 55 MWh (for 1000 PEVs in a PEVG) and a 
minimum of 9.9 MWh (for 300 PEVs) [20]. 

In Fig. 10 it is shown that the demand of any PEVG varies 
across time slots due to variation in both satisfaction parameters 
of the PEVGs and the available energy. However, the minimum 
energy demand by any PEVG is always well above its minimum 
battery requirement. For example, the minimum demand by 
PEVG 4 is 5.5 MWh in time slot 1, which exceeds the minimum 
battery requirement of a mid-size car, assuming 1000 PEVs are 
in that PEVG [41]. 

The change of utility per PEVG in a time-varying environment 
is shown in Fig. 11 for the proposed, PSO and ED schemes. 
Fig. 11 shows that the team optimal solution achieved by the 
proposed scheme leads to an improved average utility for the 
PEVG when compared to the PSO and ED schemes. However, 
the improvement varies across time slots due to variations in 
available energy and the number of PEVs in the smart grid 
network. As shown in Fig. 11, the average utility achieved per 
PEVG by our proposed scheme is, on the average, 1.6 times the 
utility achieved by the PSO scheme and 3.8 times that achieved 
by the ED scheme. 

VII. Conclusions 

In this paper, we have formulated a noncooperatibe Stack- 
elberg game to study the problem of energy trading between 
an SG and a number of PEV groups. In this game, the SG 
chooses its price to maximize its revenue whereas the PEVGs 
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Fig. 10: Average demands of the PEVGs at the GSE for the proposed 
scheme in a dynamic environment. 
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Fig. 11: Comparison of the average utility per PEVG in the dynamic 
case. 

strategically choose the amounts of energy they wish to buy 
from the grid so as to optimize a tradeoff between the benefit 
of battery charging and associated costs. We have studied the 
properties of this solution and we have shown that this game 
admits a socially optimal generalized Stackelberg equilibrium. 
We have also extended and analyzed the proposed game in a 
time-varying environment. To reach the equilibrium of the game, 
we have proposed a novel algorithm that can be adopted by 
the PEVGs, in a distributed manner. Simulation results have 
shown that the proposed approach yields improved performance 
gains, in terms of the average utility per PEVG, compared to a 
particle swarm optimization and an equal distribution scheme. 
Several future extensions can be foreseen for this work, such as 
handling rapidly changing dynamics or determining the optimal 
deployment of charging stations depending on a variety of factors 
such as the parking duration, battery state, and the location of 
the vehicles. 
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